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Navigation  and  Decision  Making  Efficiencies  under  Conditions  of 
Uncertainty. 

FA9550-04-1-0236 

Brian  J.  Stankiewicz 
University  of  Texas,  Austin 
Department  of  Psychology 

Each  and  every  day  we  make  thousands  of  decisions  in  which  our 
understanding  of  the  situation  is  not  completely  known.  Although  we 
may  have  incomplete  knowledge,  we  may  possess  knowledge  that  will  be 
helpful  in  making  a  "good"  decision.  For  example,  when  purchasing  an 
automobile,  one  does  not  know  for  certain  how  reliable  this  car  will 
be.  However,  given  the  make  and  model's  previous  track  record,  one 
might  be  able  to  generate  an  initial  estimate  its  reliability. 

However,  given  that  the  car  is  used,  there  is  some  uncertainty  about 
how  well  the  car  was  maintained  and  how  it  was  used.  One  may  consider 
purchasing  an  inspection  by  a  mechanic.  Following  the  inspection,  one 
might  be  more  confident  that  the  car  will  be  reliable— however ,  even 
with  this  additional  information  one  is  still  in  a  state  of 
uncertainty.  One  may  consider  purchasing  an  additional  inspection  to 
reduce  one's  uncertainty  even  more.  As  one  can  see,  an  individual  can 
continue  to  gather  more-and-more  information  at  a  greater  and  greater 
accumulated  cost.  However,  this  additional  information  comes  at  a  cost 
that  may  not  exceed  the  expected  gain  in  knowledge. 

The  research  in  this  program  was  interested  in  four  questions  related 
to  our  ability  to  make  decisions  with  uncertainty.  First, 
computationally,  what  is  the  optimal  decision  strategy?  To  this  end  we 
used  a  Bayesian  model  of  sequential  decision  making  with  uncertainty 
called  Partially  Observable  Markov  Decision  Processes  (POMDP) .  This 
model  provides  us  with  the  theoretically  optimal  performance  in  a 
sequential  decision  making  with  uncertainty  task.  Second,  we  were 
interested  in  characterizing  human  sequential  decision  making  with 
uncertainty.  Specifically,  we  were  interested  in  understanding  how 
well  humans  performed  relative  to  the  optimal  decision  maker.  This 
performance  gave  us  a  measure  of  human  decision  making  efficiency. 
Third,  if  we  found  that  humans  were  not  making  decisions  optimally,  we 
were  interested  in  understanding  what  sub-process  might  lead  to  this 
inefficient  behavior.  Finally,  if  people  were  sub-optimal  in  their 
decision  making  process,  we  were  interested  in  understanding  whether  we 
could  develop  decision  aids  that  would  allow  human  decision  making  to 
become  optimal,  or  near  optimal. 

To  address  these  issues  we  studied  human  spatial  navigation  when 
individuals  were  "lost"  within  a  familiar  environment  and  also  a  "Seek- 
And-Destroy"  task  in  which  the  decision  maker  had  to  locate  and  destroy 
a  target.  In  the  S&D  task,  the  decision  maker  needed  to  locate  the 
target  using  "reconnaissance"  that  would  indicate  whether  the  target 
was  observed  or  not.  However,  the  reconnaissance  was  not  perfect  in 
that  some  times  it  would  detect  the  target  at  a  location,  but  the 
target  was  not  located  there  (false  alarm)  or  the  reconnaissance  would 
not  detect  the  target  but  it  was  at  that  location  (miss) .  Furthermore, 
the  decision  maker  could  send  artillery  to  a  location  and  if  the  target 
was  there,  some  times  it  would  miss.  Furthermore,  each  of  these  actions 
(reconnaissance  and  artillery)  had  a  cost  associated  with  them.  The 


decision  maker's  task  (both  human  and  computational  model)  was  to 
destroy  the  target  and  Declare  that  the  target  was  destroyed.  If  the 
decision  maker  successfully  destroyed  the  target  when  they  declared 
they  received  a  positive  reward,  but  if  they  declared  and  the  enemy  was 
still  alive,  the  decision  maker  received  a  large  negative  reward. 

This  research  has  lead  to  three  very  interesting  and  important 
learnings : 

1.  We  were  able  to  develop  computational  techniques  to  measure  the 
theoretical  optimal  performance  in  both  the  spatial  navigation  with 
uncertainty  task  and  the  Seek-And-Destroy  task. 

a.  This  lead  to  the  development  of  a  hand-held  indoor  navigation 
aid  that  is  completely  self-contained  (Stankiewicz,  Cassandra,  McCabe  & 
Weathers,  2007) . 

2.  We  found  that  individuals  were  sub-optimal  in  their  ability  to  make 
decisions  with  uncertainty.  We  were  able  to  localize  the  sub-optimal 
behavior  to  the  inability  to  correctly  calculate  the  likelihoods  of  the 
current  true  state. 

3.  We  were  able  to  provide  users  with  a  decision  making  aid  that 
brought  their  decision  making  efficiency  (the  human  performance 
relative  to  the  optimal  performance)  from  about  50%  to  nearly  90%. 

Summary : 

In  summary,  we  have  brought  together  the  power  of  computation  by 
developing  a  computational  model  of  sequential  decision  making  with 
uncertainty  along  with  understanding  the  strengths  and  limits  of  human 
decision  making.  In  fact,  one  of  the  most  remarkable  findings  in  this 
research  program  was  that  there  is  an  interesting  symbiotic 
relationship  between  the  human  and  the  computer  in  sequential  decision 
making.  The  decision  process  in  sequential  decision  making  can  be 
broken  down  into  two  sub-tasks.  The  first  is  integrating  the  new  piece 
of  knowledge  that  was  acquired  to  update  the  likelihoods  of  the  current 
"true  state"  (e.g.,  the  likelihood  that  the  target  is  at  one  location 
or  another) .  The  second,  is  given  these  likelihoods,  what  is  the  best 
action  to  enact  at  this  time.  For  the  computer,  the  first  process  is 
quite  simple  and  on  a  typical  contemporary  computer  can  be  completed  in 
less  than  a  second.  For  many  problems  (including  those  that  were 
studied  in  this  research  project)  computing  the  best  action  given  the 
likelihoods  can  take  hours,  days  or  weeks.  However,  the  human  has  the 
opposite  problem.  We  have  difficulties  updating  the  likelihoods  of  the 
true  state,  but  given  these  likelihoods  (e.g.,  when  computed  by  the 
computer)  our  research  has  shown  that  we  can  make  near  optimal 
decisions  on  which  action  to  engage  in  next.  This  research  suggests 
that  a  very  important  cognitive  aid  for  human  sequential  decision 
making  is  one  that  updated  the  likelihoods  when  new  information  is 
received . 
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